Skill ranking system

ABSTRACT

In an example disclosed is a method of ranking social networking service member skills by retrieving from non-volatile storage a plurality of member profiles created by a plurality of members of a social networking service, running a text classification algorithm to determine which of the plurality of members possesses a skill that matches any of a plurality of provided skills and associated skill attributes; and for at least one skill of the plurality of provided skills, identifying the plurality of members that possess the skill and ranking the plurality of members relative to one another using a ranking algorithm, the ranking algorithm being based in part upon weighted interactions among the plurality of members that possess the given skill, the weighted interactions comprising endorsements between a first member who possesses the given skill and a second member who possesses the given skill.

CLAIM OF PRIORITY

This patent application claims the benefit of priority, under 35 U.S.C.Section 119(e), to U.S. Provisional Patent Application Ser. No.61/436,894, entitled “Skill Extraction and Ranking System,” filed onJan. 27, 2011 to Skomoroch, et. al, which is hereby incorporated byreference herein in its entirety.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever. The following notice applies to the software and dataas described below and in the drawings that form a part of thisdocument: Copyright LinkedIn, Co. 2011, All Rights Reserved.

BACKGROUND

A social networking service is a computer or web-based application thatenables users to establish links or connections with persons for thepurpose of sharing information with one another. Some social networksaim to enable friends and family to communicate with one another, whileothers are specifically directed to business users with a goal ofenabling the sharing of business information.

SUMMARY

In an example, disclosed is a method of assigning a skill rank on asocial networking site by retrieving from non-volatile storage aplurality of member profiles created by a plurality of members of asocial networking service, running a text classification algorithm todetermine which of the plurality of members possesses a skill thatmatches any of a plurality of provided skills and associated skillattributes, and for at least one skill of the plurality of providedskills, identifying the plurality of members that possess the skill andranking the plurality of members relative to one another using a rankingalgorithm, the ranking algorithm being based in part upon weightedinteractions among the plurality of members that possess the givenskill, the weighted interactions comprising endorsements between a firstmember who possesses the given skill and a second member who possessesthe given skill.

In another example, disclosed is a system with a retrieval module toretrieve a plurality of member profiles created by a plurality ofmembers of a social networking service, a tagging module executable onone or more computer processors to run a text classification algorithmon the plurality of member profiles to determine which of the pluralityof members possesses a skill that matches any of a plurality of providedskills and associated skill attributes, and a ranking module configuredto: for at least one skill of the plurality of provided skills, identifythe plurality of members that possess the skill and rank them relativeto each other using a ranking algorithm, the ranking algorithm beingbased at least upon weighted interactions among members that posses thegiven skill, the weighted interactions comprising endorsements between afirst member that possesses the given skill and a second member thatpossesses the given skill.

In yet another example, disclosed is a machine-readable storage mediumincluding instructions, which when executed on the machine, causes themachine to retrieve from non-volatile storage a plurality of memberprofiles created by a plurality of members of a social networkingservice, execute, a text classification algorithm to determine which ofthe plurality of members possesses a skill that matches any of aplurality of provided skills and associated skill attributes, and for atleast one skill of the plurality of provided skills, identify theplurality of members that possess the skill and rank the plurality ofmembers relative to one another using a ranking algorithm, the rankingalgorithm being based in part upon weighted interactions among theplurality of members that possess the given skill, the weightedinteractions comprising endorsements between a first member whopossesses the given skill and a second member who possesses the givenskill.

These examples can be combined in any permutation or combination. Thissummary is intended to provide an overview of subject matter of thepresent patent application. It is not intended to provide an exclusiveor exhaustive explanation of the invention. The detailed description isincluded to provide further information about the present patentapplication.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows one example method of the current disclosure.

FIG. 2 shows an example method of obtaining a standardized list ofskills.

FIG. 3 shows an example method of seed phrase extraction.

FIG. 4 shows an example method of seed phrase disambiguation.

FIG. 5 shows an example association matrix.

FIG. 6 shows an example method of phrase de-duplication.

FIG. 7 shows an example method of phrase validation.

FIG. 8 shows an example method of skill tagging.

FIG. 9 shows an example method of calculating member behavior metrics.

FIG. 10 shows an example method of ranking members.

FIG. 11 shows an example skill graph.

FIG. 12 shows additional steps in some examples of ranking members.

FIG. 13 shows an example social networking site customization system.

FIG. 14 shows an example social networking system.

FIG. 15 shows a first screen capture of an example of aspects of thecurrent disclosure.

FIG. 16 shows a second screen capture of an example of aspects of thecurrent disclosure.

FIG. 17 shows a third screen capture of an example of aspects of thecurrent disclosure.

FIG. 18 shows an example computer system.

In the drawings, which are not necessarily drawn to scale, like numeralsmay be associated with similar components shown in different views. Likenumerals having different letter suffixes may represent differentinstances of similar components. The drawings illustrate generally, byway of example, but not by way of limitation, various embodimentsdiscussed in the present document.

DETAILED DESCRIPTION

In the following, a detailed description of examples will be given withreferences to the drawings. It should be understood that variousmodifications to the examples may be made. In particular, elements ofone example may be combined and used in other examples to form newexamples.

Many of the examples described herein are provided in the context of asocial or business networking website or service. However, theapplicability of the inventive subject matter is not limited to a socialor business networking service. A social networking service is an onlineservice, platform or site that allows members to build or reflect socialnetworks or social relations among members. Typically, members constructprofiles, which may include personal information such as name, contactinformation, employment information, photographs, personal messages,status information, links to web-related content, blogs, and so on.Typically, only a portion of a members profile may be viewed by thegeneral public, and/or other members.

The social networking site allows members to identify, and establishlinks or connections with other members in order to build or reflectsocial networks or social relations among members. For instance, in thecontext of a business networking service (a type of social networkingservice), a person may establish a link or connection with his or herbusiness contacts, including work colleagues, clients, customers, and soon. With a social networking service, a person may establish links orconnections with his or her friends and family. A connection isgenerally formed using an invitation process in which one member“invites” a second member to form a link. The second member than has theoption of accepting or declining the invitation.

In general, a connection or link represents or is otherwise associatedwith an information access privilege, such that a first person who hasestablished a connection with a second person is, via the establishmentof that connection, authorizing the second person to view or accessnon-publicly available portions of their profiles. Of course, dependingon the particular implementation of the business/social networkingservice, the nature and type of the information that may be shared, aswell as the granularity with which the access privileges may be definedto protect certain types of data may vary greatly.

In the context of business social networks, users often may submit alist of skills that they possess as part of their member profiles. Otherusers, advertisers, and businesses may then use these skill lists toascertain what a particular member is good at or interested in. Theinherent problem with using member-submitted skills is that it isentirely subjective and prone to fraud. Thus a member may present him orherself as having a skill they do not possess. In addition, even thougha member may possess a certain skill, there is no indication that theyare proficient in that skill.

The present disclosure describes a method, system and product foridentifying a set of standardized skills from member profiles of asocial or business networking service. The list of standardized skills,along with information in a member profile section of the socialnetworking service may be used to identify members of the socialnetworking service that possess one of those identified skills Membersidentified as possessing a given skill may be ranked relative to oneanother with respect to the given skill based upon various implicit,explicit, internal and external factors. The skills and rankings may beused to deliver content and customization to those members and others.

FIG. 1 presents a high level view of the method according to one exampleimplementation. In step 1010, according to one example implementation,the system may obtain or generate a standardized list of skills withwhich to rank users relative to one another. With some embodiments,these skills may include specific skills such as the ability to programin a particular programming language, such as Java or C++, or broaderskills, such as the ability to program a computer, or specialized skillssuch as programming web-based applications. While reference is made toskills in the present disclosure, it will be understood by those skilledin the art with the benefit of the present disclosure, that thetechniques taught herein are applicable to other concepts.

The standardized list of skills may be obtained by utilizing apre-determined list of skills. In one example, the predetermined list ofskills may be manually generated, but in other examples thepre-determined list of skills may be automatically generated. In stillother examples, the list of standardized skills may be created byprocessing member profiles of a social or business networking service.In some examples, this processing can be done automatically using acomputing system or other machine. In yet other examples, thisprocessing could be manually accomplished. In some examples, a skillssection of a member profile of a social networking service may be used.The skills section of the member profile may be a free-text section thatallows users to freely type in skills they possess, this information isgenerally referred to as unstructured information. Alternatively in someother examples, the member profile skills section may be implemented asa list that allows users to choose a skill based upon structured datasuch as a predetermined listing of skills, or in other examples, theskills section may be implemented as some combination of unstructureddata such as free-text and structured data such as a pre-determined listselection.

In step 1020, the system may then determine, or “tag” members of thebusiness or social networking service who possess one of thestandardized skills. In some examples, “tagging” can include associatingan item of meta-data with the member profile of the member who is taggedthat indicates that this member possesses a certain skill. In otherexamples, information about which skills a member possesses may beincluded directly in the member's profile. In one example, members aretagged based upon the information in their member profile in a socialnetworking service. In other examples, members may select skills theyare proficient in from a list of the standardized skills. In still otherexamples, other members may determine a particular member's skills byuse of feedback mechanisms such as surveys.

In step 1030, the system may then rank all the members who have beentagged as possessing certain skills relative to one another to achieve askill ranking. In one example, the skill ranking is based uponactivities that occur on the social networking service. Thus forexample, a member who has many connections to other members who alsopossess the skill would be more highly ranked than other members whohave fewer connections to other members who possess the certain skill.In other examples, these connections may be weighted such that aconnection to another member who is highly rated for that skillincreases the member's ranking more than a similar connection with alower ranking member. In still other examples, other factors are used torank members in conjunction with, or instead of, activities on thesocial networking service. In some examples, authorship of scholarlyarticles on or about the skill is considered. Authorship or editorshipof articles, websites, blogs, Wikipedia entries, or discussion groups orforums may also be considered in other examples.

In step 1040, the rankings and tagging of skills may be used to providevarious customization and services to the social or business networkingservice and its various members. In some examples, members may beprovided their rankings. In still other examples, lists may be createdand published. In yet other examples, companies and geographical areasmay also be ranked using the ranking of individuals who work, live, orare from specific companies or locations. In still another example,recommendations may be generated to members on how to improve theirskill ranking.

Obtaining a Standardized List of Skills

Turning now to FIG. 2, one example method of obtaining a standardizedlist of available skills is shown. In step 2010, seed phrases may beextracted from text contained in member profiles of members of thebusiness or social networking service. Seed phrases in one example areone or more words that represent a possible skill. The seed phrases maybe individual words such as “Java” or phrases of words such as “java.net,” or “search and seizure.” In one example, the seed phrases may beextracted from a skills section of the member profiles, but in otherexamples, seed phrases may be extracted from other sections of amember's profile. In one example, the skills section of a member'sprofile is a free text (e.g., unstructured) section that allows membersto type in any skills they feel they possess. In some examples, allmember profiles of a social networking service are used to gather seedphrases, but in other examples, only a subset of all member profiles maybe used. For example, the system may only extract seed phrases fromprofiles of members in a particular industry, in a particular geographicregion, or who work for a particular company.

Along with gathering the skill seed phrases, context information, or“meta data,” may be gathered. One such item of meta data may includeco-occurent phrases. Co-ocurrent phrases are words or phrases that occurin the same member profile as the seed words or phrases and are used ina later processing operation as one way of ascertaining an intendedmeaning of a seed phrase. A given phrase may be a co-occurrent phrasefor a particular skill seed phrase, and may be a skill seed phraseitself. Additionally, this meta data may include other information inthe member profile of the members in which the seed phrase exists,including a member's reported industry, institution, employer, projects,geographic location, group membership, and the like.

FIG. 3 presents one example of the operations performed to extract seedphrases from member profiles. In step 3010 member profiles from a socialnetworking or business networking site are retrieved from an electronicstorage area. The electronic storage area may include computer memory,both non-volatile and volatile, a computer database, another computersystem, or the like. In some examples, all member profiles areretrieved, but in other examples only certain member profiles may beincluded in the skill seed phrase extraction. These selected memberprofiles may be selected based on a variety of factors. Some factors mayinclude a predetermined list of members, members listing an associationwith a particular school, organization, work environment, workplace,geographic location, skills listed, or member popularity.

In step 3020, the specialties section is retrieved from the memberprofiles. For instance, with some embodiments, the specialties sectionis that portion of a member's profile that stores the member'sself-described or selected skills, or specialties. Each specialtiessection may then be tokenized based upon commonly used delimiters suchas a comma, slash, carriage return, conjunctive or disjunctive words(“and,” “or”), and the like. Tokenization is the process of breaking astream of text up into words, phrases, symbols, or other meaningfulelements called tokens. Thus for example, a member's specialties sectionof a profile might contain the text “construction industry, housing anddevelopment, foundations/support.” The system may initially tokenizethis into “construction industry,” “housing”, “development,”“foundations,” “support.” Once the text is tokenized, the systemcalculates the number of times a particular token is found in thespecialties section of the member profiles of the system. The memberspecialties section is used herein for illustrative purposes, and asalready stated, other sections may be used to establish the skill seedphrases.

In some examples, certain aspects of the present disclosure, includingtokenization may be done in parallel using a batch processing systemover a distributed computer system. In some examples, this distributedcomputer system may be managed by Apache Hadoop, which is a softwareframework that supports data intensive distributed applicationsdeveloped by the Apache Software Foundation, Inc. In some examples,certain aspects of the present disclosure, including tokenization may beimplemented by the MapReduce software method which is a framework forprocessing huge datasets on distributable problems using a large numberof computers (or nodes) which are referred to as a cluster. MapReduce isdescribed in U.S. Pat. No. 7,650,331 issued to Dean, et. al. andassigned to Google Inc., of Mountain View, Calif., which is herebyincorporated by reference in its entirety. In MapReduce, there are twophases: the map phase and the reduce phase. In the “map” phase, “chunks”of data are assigned to different servers which then process the dataaccording to a defined algorithm and return a result. The servers maybreak up the data into even smaller chunks and assign each smaller chunkto a map process running on the server, where many map functions mayexecute on a single server. The results from all the map processes arethen aggregated according to a predefined process in the “reduce” phase.

In the case of the tokenization in step 3020, the data may be chunkedfor the map phase into any portion or subportion of the input data usedto create the standardized list of skills. In some examples, the chunksmay include a plurality of profiles, a single profile, sections ofprofiles, or even sections of text from a portion of a profile, forexample, the specialties or skills section. The map processes may thentokenize the given data chunk by parsing the given data chunk andsplitting it into words or phrases based upon the delimiters used. Eachmap process then returns each token to the reduce process. The reduceprocess may then count the number of times a particular token has beenpassed back by all the various map processes, establishing a tokenfrequency. In some examples, this map-reduce frequency calculation maybe done multiple times. The first passes may use a minimal set ofdelimiters whereas additional passes may add additional delimiters. Thismay result in establishing frequency statistics for both longer phrases(“search and seizure”) as well as constituent individual words(“search,” and “seizure), which in some examples may be used in laterstages.

While distributed computing methods using MapReduce are describedthroughout this disclosure, it will be appreciated by a person who isskilled in the art with the benefit of the present disclosure that othermethods are possible. For example, a single computer system may do allthe processing described as opposed to a distributed computing system.Also, instead of MapReduce, other solutions may be used, including butnot limited to, the use of “if-then” and “for loop” programmingtechniques to iterate over all the member profiles and skills sectiontext in order to tokenize and count token frequency, and perform othermethod steps of the present disclosure. In addition, other distributedcomputing solutions may be utilized apart from Hadoop. Alternativedistributed computing approaches may be employed such as Message PassingInterface (“MPI”) or a cluster of workers with a single master node topartition out parsing tasks.

In step 3040, the frequency of token occurrence information may be usedto determine whether two different tokens correspond to a specific skillphrase and therefore should not be separated by the tokenization. Forexample, the phrase “search and seizure,” might be broken up in step3020 into “search” and “seizure,” however the skill phrase “search andseizure,” would be best kept together as it likely refers to one skill.Some skill phrases such as “C++ and Java” should be broken apart into“C++,” and “Java,” as those are considered separate skills. In someexamples, whether or not to split the seed phrases may be determined bycalculating whether any of the component tokens occurred individuallyless often than the compound phrases. If not, then the component tokenswill be kept separate, otherwise they will be combined. Thus forexample, frequency information for “search,” “seizure,” and “search andseizure” may be calculated. If “search” appeared 5 times and “seizure”appeared 3 times, but “search and seizure” occurred 10 times, then theskill seed phrase may be the compound phrase “search and seizure.”

In step 3050, this first pass data may be fed back into the system toscan member profiles again to determine a count of how many times eachphrase occurs in the member profiles. In some examples, this may be doneusing MapReduce and Hadoop as in step 3020. In this case however,instead of splitting at the selected delimiters automatically, thesystem may use the analysis performed in step 3040 to come up with arefined splitting algorithm. Thus, for example, instead of splitting“search and seizure,” the system may treat it as a single phrase inproducing a frequency count if the analysis in step 3040 indicates itshould be treated as such. In some examples, this may be an iterativeprocess and the data may be fed back into scan member profiles again,each time with a refined splitting algorithm until the list of skillsconverges.

In step 3060, certain non-skill seed phrases may be removed from furtherconsideration. Thus phrases clearly not relating to skills may beremoved. For example, phrases corresponding to certain categories oflanguage not likely to be skill related may be removed. In someexamples, articles, prepositions, verbs, nouns, or any combination maybe removed. In some examples, phrases that may be inappropriate,offensive or too graphic may be removed. Various methods may be used toachieve this, including submission of the phrases to crowd-sourcingjobs, dictionaries, or blacklists A “blacklist” is a list that containscommon non-skill phrases. If a skill phrase is on the blacklist, it maybe removed from further processing. In some examples, this operation maybe done prior to tokenization after the member profile section is readfrom storage.

In step 3070, in some examples, statistically insignificant seed phrasesmay be removed from further consideration. Thus if the frequency ofoccurrence of a skill seed phrase is below a threshold, that particularskill seed phrase may be removed from further consideration. Thus, forexample, if only one profile out of thousands contains the skill seedphrase, that seed phrase may not be particularly interesting. Thisallows the size of the skill seed phrase list to be reduced. Thethreshold may be a predetermined value that indicates a minimum numberof times the phrase must occur (e.g., 10 times) to be included, or apredetermined percentage (e.g., it must be included in 0.5% of thescanned member profiles), or some other dynamic algorithm.

In 3080, in some examples, a spelling checker and correction algorithmmay be used to find and correct spelling deficiencies in the skill seedphrase list. This is to shrink the size of the skill seed phrase listand make the task of de-duplication easier in later stages byeliminating improperly spelled variants. This may be desirable for skillseed phrases in which misspellings are common.

In step 3090, the resulting list of skill seed phrases not removed fromconsideration may be output and may be called the “Seed PhraseDictionary.”

In examples in which the set of standardized skills is determined basedupon a free-text area of a member's profile, the various collected seedphrases may be ambiguous. That is, phrases may have more than onemeaning, or “senses,” and subsequently refer to different skills. Forexample, the text “search,” in a user's skill section of a profile, mayrefer to a law enforcement context, or it may refer to an internetsearch context, or it may be a talent search context.

Returning now to FIG. 2, because of this problem, in some examples, thenext step in obtaining a standardized list of skills may be phrasedisambiguation carried out in step 2020. In phrase disambiguation, thelist of skill seed phrases may be expanded to capture the different“senses” of the phrases. “Senses” are different meanings of a givenphrase. So, for example, if the list of skill seed phrases initially is“search,” and information is found in the member profiles to suggestseveral different senses of “search,” then the list of skill seedphrases may be expanded to include all or some of the particular senses.Additionally, the skill seed phrases may be annotated to identify thesense. Thus the list of skill seed phrases might expand from one phraseto three (i.e., “search” becomes “search” in the computer science sense,“search,” in the law enforcement sense and “search” in the recruitingsense).

FIG. 4 shows one example implementation of a disambiguation algorithm.In step 4010, an association matrix may be built by reprocessing theskills section of the member profiles again. The MapReduce functionalitymay be programmed to emit a count of a co-occurrence of each pair ofterms in the seed phrase dictionary for every member profile. Aco-occurrence is an instance where two seed phrases occurred in the samemember profile. Thus if there are ten terms in the seed phrasedictionary, the association matrix may be a ten-by-ten matrix, each rowand column intersection in the matrix corresponding to a count of thenumber of times the pair of dictionary seed phrases occurred in thescanned member profiles. FIG. 5 depicts a basic example of anassociation matrix that shows the co-occurrence of six dictionary seedphrases. Thus, for example, the term “Search and Seizure” occurred inthe same profile as the term “Law Enforcement” 15 times, whereas itnever co-occurred with the term “Computer Software.”

In step 4020, a probability analysis may be run using the associationmatrix to determine, based on a given skill seed phrase, what the likelyco-occurrent phrases are. This may be expressed as a probability thatgiven a skill seed phrase, a different phrase will be in co-occurrence.Thus, in FIG. 5, the probability that “Search and Seizure” was presentin the same profile as “Law Enforcement” will likely be very high. Insome examples, this algorithm may include various similarity metricslike Jaccard Similarity or Term Frequency Inverse Document Frequency(TFIDF).

In step 4030, the probabilities may be used to “cluster” the variousrelated seed phrases into senses using the calculated probabilities. Theseed phrases may be clustered based upon the probability that certainco-ocurrent terms of the skill seed phrases will occur with otherco-occurrent terms. Thus for example, if “search” has a high probabilityof being co-occurrent with the skill seed phrases “law enforcement,”“fbi”, “computer programming,” and “Java,” the system may use theco-occurrent information between those likely co-occurrent phrases todetermine “clusters” of “search.” Thus for example, if “law enforcement”had a high probability of being co-occurrent with “fbi” and “fbi” had ahigh probability of being co-occurrent with “law enforcement,” but NOT“computer programming,” and NOT “Java,” then one cluster may be “search,law enforcement, fbi.” If Java and computer programming are likelyco-occurrent phrases between themselves, then another cluster could be“search, Java, computer programming.”

To perform this clustering, an expectation maximum algorithm may beused. For example, an algorithm such as K-means may be used.Co-occurrent phrases may be compared with each other pairwise in thespace of all frequently co-occurring or similar phrases for theseed-phrase. Rows of this distance matrix may then be clustered, andclusters may be merged or split as needed until a converged set ofdisambiguated phrase senses emerge.

In step 4040, the top industry information for each cluster may becomputed. This may be done by processing the member profiles usingHadoop and MapReduce again. In this case, the member profiles may besearched for the various dictionary skill seed phrases. Upon finding adictionary skill seed phrase, the system may read the industryassociation stored in the member profile. The industry association insome examples is a member-selected industry association. In someexamples, the member may select from a predetermined list of industries.In other examples, the industry association may be a free form textassociation. The clusters may then be analyzed to determine the topindustries associated with the skill seed phrases in that cluster. Thisinformation may then be stored and used in later stages.

The output of the disambiguation may result in a list of disambiguatedskill seed phrase clusters annotated with industry information.

Because the member profile section may contain typos, or differentspellings or words to describe a single skill (such as “java net” vs.“java.net”), and because the result of the disambiguation may sometimeslead to skill duplications the disambiguated skill seed phrases may needto be de-duplicated. De-duplication is the process by which duplicateskill seed phrases are removed from further consideration.

Continuing with FIG. 2, in step 2040, the disambiguated skill seedphrases may then be de-duplicated. FIG. 6 shows one example method forde-duplicating the seed phrases. In step 6010, a Wikipedia or otherinternet search query may be generated using, in some examples, theskill seed phrase, co-occurrent phrases, and/or industry information. Insome examples, only the disambiguated skill seed phrase itself is used.In other examples additional information such as co-occurrent phrases,and/or industry information may be used. This internet query may beconstructed as merely a concatenation of all the information regardingthe skill cluster, such as for example: “search search and seizure lawenforcement FBI police sheriff DEA drug enforcement agency.” Some otherexamples may use Boolean operators such as ‘and’, ‘or’, ‘not’, or ‘xor’between the various pieces of the search query. Alternatively, the querymay be compared against text collections or web pages stored offlineusing an inverted index or text similarity metrics applied against adocument collection.

When the internet web query is executed in an internet or other searchengine a list of internet web pages representing a list of possiblematches for that query may be produced. In some examples, the internetsearch engine may be an internet-wide search engine such as Google, runby Google Inc. of Mountain View, Calif. In some examples, the searchengine may be a site-specific search engine, such as the search engineof Wikipedia. Wikipedia is a searchable, online, collaborativeencyclopedia project supported by the Wikimedia Foundation, a FloridaCorporation headquartered in San Francisco, Calif. In some examples theinternet web query, when executed in Wikipedia, may return a list ofWikipedia entries corresponding to pages of the Wikipedia.

At step 6020, the skill seed phrase, the co-occurrent phrases, theindustry information, and the Wikipedia or other internet search enginequery may be passed to a crowdsourcing job of a crowdsourcingapplication. Crowdsourcing is the act of outsourcing tasks to anundefined, large group of people or community through an open call. Inone example implementation of crowdsourcing, a problem or task isbroadcast to a group of individuals looking for tasks. Those with aninterest in solving the problem decide to accept the task. Once asolution is found, the solution is passed to the party who posed theproblem or task. Usually, a small payment is then provided to the partywho solved the problem by the party who posed the problem. One examplecrowdsourcing implementation is Mechanical Turk™ run by Amazon.com, Inc.of Seattle, Wash., in which Amazon provides a marketplace in whichbusinesses post tasks that need completion and offer a reward forcompleting the task. The reward may be any monetary value, but generallyis a small reward of a few pennies per task. Individuals looking fortasks then may accept and complete those tasks to gain the reward.

In one example, the job submitted to the crowdsourcing application mayask the worker to pick the internet web page from the list of internetweb-pages returned by the search query that corresponds to theparticular skill seed phrase. Thus, in one example, if the skill seedphrase is “search,” with a related concurrent phrase “legal,” the searchquery might be “search legal,” and may return Wikipedia results such as:

-   -   “search and seizure”    -   “Legally Blonde—The Musical: The Search for Elle Woods”    -   “JustCite”    -   “LawMoose” . . . .

In that example, the worker would pick “search and seizure” to signifythat the particular skill relates to searches and seizures of lawenforcement. Other similar skills should return the same page. In thisway, in step 6030 duplicate skills may be determined based on commonweb-pages returned by the crowdsourcing workers.

In some examples, a single skill seed phrase may be submitted tomultiple workers. This is to ensure the quality of the worker responses.Each worker would then make their selections, and various algorithms instep 6030 may be used to pick the result if the workers come back withdifferent results. One example algorithm may be a majority algorithm,whereby the page selected by the majority of workers will be selected.Other example algorithms use a consensus pick.

Other examples of de-duplication may be used, such as using thecrowd-sourcing worker to sort a list of skill seed phrases to findduplicates using just the skill seed phrases and the co-occurrentphrases and associated industry information. Other implementations mayinclude using the crowdsourcing worker to find a Wikipedia page or otherwebpage that describes the particular skill without first presenting theworker with a constructed query.

Once the disambiguated skill seed phrases are de-duplicated, the phrasesmay then be validated in step 2050 of FIG. 2. One example validationmethod is shown in FIG. 7. In step 7020, the Wikipedia or other URL isvalidated. In one example, this may be validated by anothercrowdsourcing job that simply asks the worker to determine if the URLreturned correctly corresponds with or describes the skill phrase. Otherautomatic algorithms may be used, including examining the frequency withwhich the phrases and terms in the skill seed phrase and related metadata (such as the common co-occurrent phrases and industry information)appear in the returned website. A low frequency may indicate anincorrect website that may be flagged for later scrutiny.

In step 7030, the returned URL or Wikipedia entry may be scraped toascertain more information, such as more related phrases and industries.The result may be added to the skill phrase meta-data and may result ina standardized list of skills and related meta information about thoseskills that may be used to “tag” individuals with those skills. Asalready explained, in some examples, the skill phrase meta data maycontain co-occurrent phrases, industry information, and the informationscraped from the returned URL.

Referring back to FIG. 2, in step 2060, additional attributes may becalculated by running the member profiles back through the profileprocessing. Such attributes may include calculating the top industry,related phrases, and other statistical information about the skill seedphrases. This extra step may be done in some embodiments, rather thancollecting this information along with other processing steps abovebecause the skill phrases may be constantly changing. Thus because ofthe de-duplication above, the statistics kept (i.e. top industry, etc. .. . ) may need to be updated to reflect this de-duplication.

Tagging and Ranking Members with Skills

Returning now to FIG. 1, once a standardized list of skills and possiblyother information such as related terms and industries is determined,members with those skills may be determined in step 1020.

FIG. 8 shows an example method of “tagging,” or identifying members thatpossess one of the skills in the standardized list of skills. In step8010, a set of member profiles may be retrieved from a database or othercomputer memory. In step 8020 information from the member profiles maybe retrieved. In some examples, the information may be the text or asegment thereof of the member specialties section of the member profile.In other examples, the information may also include details such asindustry information, company information, or any other piece ofinformation from the member profile including member status updates. Inyet other examples, external information from other internet sites maybe gathered based upon any link found in a member profile. For example,a website or a blog listed on a profile may be scraped for content thatis then tokenized for input into the tagging algorithms. In someexamples, if the external site contains another link, that link may thenbe processed as well.

In step 8030 an algorithm may be used to determine whether, based on allthe evidence, a particular member is likely to have a particular skill.In one example, the algorithm may be a Bayesian text classifier. In someexamples, there may be a classifier for each skill seed phrase sensethat is trained with the skill seed phrase dictionary, related phrases,frequency counts, and/or industry information. In this example, thetokenized phrases of member profile text and external data is fed in asevidence (e.g., input to the algorithm) and the output of the Bayesianclassifier is a probability that a particular member possesses aparticular skill. Other example algorithms include for example, a neuralnetwork, term frequency computations or any text based classificationalgorithm.

In step 8040, the probability produced by the text classificationalgorithm at step 8030 may be run through another algorithm to determinewhether or not the member should be tagged with a specific skill. In oneexample, the algorithm may be a threshold value. For example, thethreshold could be set so that if the classification algorithm producesa 70% chance that the particular member possesses the given skill, thenthe member may be tagged as having the particular skill. In otherexamples, the threshold may vary depending on the application. Forexample, “tagging” a user with a particular skill for ranking purposesmight demand greater certainty than “tagging” a user for advertisingpurposes. Thus the threshold may be dynamically adjusted based onintended uses of the skill information.

In some examples, tagging may be indicating in some fashion in themember's profile that this member possesses the particular skill. Forexample, meta data representing the skills possessed by the member maybe stored in association with a member's profile. In other examples,tagging may be achieved through keeping a separate list of members thatpossess the particular skill. Tagging may be accomplished through anymeans in which the system may store an indication of what particularmembers possess a particular skill or skills. Tagging may also includestoring the probability generated in step 8030.

The result of step 8040 is that members possessing a certain skill areidentified and tagged at step 8050. The resulting list of members thatpossess a certain skill may be a community, or network of individualswith that skill. This may be referred to as a skill community.

After members with a particular skill have been identified, or “tagged,”those members may be ranked relative to one another. Referring back toFIG. 1, this is step 1030. FIG. 9 shows, in one example implementation,a preliminary step in ranking members.

FIG. 9 shows a collection of member behavior metrics that may be usefulin calculating a member's rank in a particular skill. In step 9010member profiles may be retrieved. In step 9020 member behavior metricsmay be collected, derived or calculated. The member behavior metrics mayinclude or be based on information concerning any activity generated byor about the member. In some examples, this may include informationabout events a member has attended, searches a member has performed,member industry information, how many years of experience the memberhas, how selective the member is on acceptance of invitations, and thelike. In some examples, the behavior metrics may also includeendorsement information. The endorsement information includesinformation relating to an indicator of support or acceptance betweenindividuals. This endorsement information may be not only from thesocial networking site itself, but also endorsement information fromexternal sites. Endorsements may include data such as profile pageviews, various follow, mention, and messaging actions on socialnetworks, favorites, shares, upvotes, invitations to connect, acceptanceof connections, emails, company relationships, group memberships,location proximity, bookmarks, referrals to that member and from thatmember, and recommendations. Some example endorsements that may be usedinclude a follower relationship on the microblogging service Twitter,operated by Twitter, Inc. of San Francisco Calif., connections onLinkedIn, run by LinkedIn, Inc. of Mountain View, Calif., friendrelationships on Facebook, of Palo Alto, Calif., MySpace of BeverlyHills, Calif., and run by News Corporation, connections on github, runby GitHub, Inc. and the like. In some examples, the endorsement ormember behavior activity information may also include frequencyinformation that determines the frequency of a particular connection orbehavior.

FIG. 10 shows an example ranking algorithm that may be used to rankmembers relative to one another. In step 10010 the community of memberswith a particular skill may be ascertained based on the earlier tagging.In step 10020, a directed skill graph may be built using the variousmembers tagged with the particular skill as nodes and edges representingthe various behavior and endorsement metrics calculated in FIG. 9 foreach member that apply to the relationship between each of the membernodes. Examples include, but are not limited to, connections, profileviews, Twitter followership, message sending between the member nodes,referrals, recommendations, and the like. Each edge may then be given aweight depending on the type of edge that is represented. Thus, in oneexample, a connection in the social network may be weighted more heavilythan a page view. Initial scores may then be computed in step 10030based on the edge weights. In some examples, the weights of the edgesare added together to form the initial score. In other examples, otheralgorithms may be used.

In step 10040, the properties of each node may be examined to adjust theweight of each edge, and thus the initial score. For example, if twomembers are connected with an edge, but one member never views the othermember's page, then that edge may be given less weight. This indicatesthat the edge between the members may not be that strong because perhapsa user felt socially obligated to be polite and make a connection ratherthan decline an invitation. In general, in some examples, if a node hasvery low behavioral metrics that are representative of memberinteractions with that member (such as such as profile views, messages,and connection information), the value of the weighting of those edgesto and from those nodes may be reduced. Alternatively, in some examples,weightings may be increased or decreased based on the member behavior orendorsement metrics. In some examples, the weight for a particular edgemay be increased or decreased based on the initial score of the nodewith which that edge is associated. Additionally, in some examples,scores may be increased or decreased based on employment, industryassociations, location of residence, location of employment, education,and other factors and attributes. This may be based upon, in someexamples, the statistics collected and calculated in step 2060 of FIG.2. Thus for example, if a particular individual worked for, or followeda particular company that was important for a particular skill, thatparticular member's scores may be increased.

An example skill graph is shown in FIG. 11. In FIG. 11, five users arerepresented as nodes in the graph. An arrow line represents anendorsement from one member to the other. The recipient of theendorsement is awarded 10 points. A dotted line indicates acceptance ofthe endorsement and increases the sender's score by five points. Aflared arrow indicates a page view and is worth one point for the memberwhose profile or homepage was viewed. Once the edge connections aremade, the scores may be calculated. Other scores for each edgeconnection type may be used; the scores of ten, five, and one for thevarious behavior metrics are exemplary only. While a simple additionalgorithm is demonstrated in FIG. 11, additional algorithms may be usedto calculate the scores.

Additionally, once the algorithm has been run once, the algorithm may bere-run, and the strength of the weights to give the various edges may beadjusted based upon the skill rank of the user to which the connectionpertains. For example, based upon the initial run presented in FIG. 11,since user 1 has the highest skill level (13), those with connectionswith user 1 may have the weight of those edge connections increased.Thus an edge connection with user 1 may be worth 11 points as opposed to10 points in one example. This algorithm may be run until the scoresconverge. In some examples, eigenvalue centrality algorithms may be usedto rank the graph nodes including degree centrality, betweennesscentrality, closeness centrality, and eigenvector centrality. Thisalgorithm in another example may incorporate principles of thePageRank®, or HITS (Hyperlink-Induced Topic Search) link analysisalgorithm. The PageRank® algorithm is fully described in U.S. Pat. No.6,285,999 assigned to Stanford University which is hereby incorporatedby reference in its entirety. The HITS algorithm is fully described inU.S. Pat. No. 6,112,202 assigned to International Business Machineswhich is hereby incorporated by reference in its entirety.

After the scores converge, in some examples, the scores may be modifiedeven further, taking into account certain other attributes. FIG. 12shows an example method of calculating these other factors. In step12010 commonalities may be found between members with a particularskill. These commonalities may include identifying which companiesemploy high ranking members, which schools high ranking members havelisted as attending, which geographical locations high ranking memberslive or work in, which related groups or other social networks highranking members belong to, and the like. Each of these factors then maybe fed back into the ranking process at 12020, such that members ofthese common groups may have their scores increased or decreased. Atstep 12030 the member score may then be recomputed using thesecommonalities by rerunning the algorithm until the scores re-converge.While some of these same factors may have been used in step 10040 ofFIG. 10, this step is more accurate as it is based on an actual rankingof the nodes and not just skill seed phrase statistics.

In still other examples, a high ranking in a related skill may be usedto increase a member's rank in a particular skill. For example, a highranking in a skill such as “C++” may increase a member's ranking in a“Java” skill. This may be done by using the phrase attribute statisticscollected after phrase validation in the obtaining skills portion, or itmay be based on rankings of individuals. For example, the system mayexamine individuals highly ranked in a particular skill and find outwhich other skills those individuals are most commonly highly rated in.For example, if most of the highest rated people for the skill“accountant,” also have a high skill level for “tax preparation,” thenan individual who has an “accountant” skill may have their “taxpreparation,” skill score increased.

Customization Based on the Skill Rankings

Referring back to FIG. 1, once the skill rankings are assigned, variouscustomizations and application of the rankings may be achieved in step1040. The skills customization methods and processes which createcustomized features for the social networking service may be implementedseparately, in one example—in a separate skills section of the socialnetworking service, or may be integrated into the social networkingsystem, or any combination of the two. Thus the customizations describedmay be added onto existing sections or pages of the social networkingservice, or may be a new, stand-alone section, application, or website.These skill customizations may take the form of HTML, text, JavaScript,FLASH, Silverlight, or any other type of textual, audio, video,audiovisual or other content. Customizations may be delivered as part ofthe social networking service or as part of some other stand aloneapplication.

In some examples, members may be shown their rankings for each skillthey are tagged as having, or in other examples, only certain skillswill be shown. In other examples, members may be shown other member'srankings. In some examples, an entire list of all members ranked may beshown. In yet other examples, a top-ten, a top-fifty, or some othersegment of the rankings may be shown. In yet other examples, members mayview information about rankings for skills they are not tagged ashaving.

In still other examples, a company rank may be computed using the scoresof the individuals that represent themselves as working for thatparticular company. As already noted, this company score may thenincrease the scores of the individuals that represent that they work forthat company. This company rank or score may be displayed to interestedusers of the social networking service.

In still other examples, a location or geographic rank may be computedusing the scores of the individuals that represent themselves as workingor living in that area. As already noted, this geographic rank may thenincrease the scores of the individuals that represent that they lived orworked in that geographic region. In other examples, the geographic rankmay be computed based upon a company rank using the locations of thecompanies. Thus geographic locations with more highly ranked companieswill be ranked higher. This location or geographic rank may be displayedto interested users of the social networking service.

These rankings may be displayed to users to customize the userexperience. In some examples, the rankings may be displayed staticallyin time, but in other examples, the rankings may show trends. Thusgeographic trends, company trends, time trends, and other skill trendsmay be constructed.

In yet other examples, members may be given recommendations on how toimprove their rankings in a particular skill. These recommendations maybe based upon the calculations used to arrive at the user's ranking. Forexample, the ranking may advise a user to seek out another member andconnect with them, or advise them to attend a particular school oruniversity, or publish a paper or write a blog on a particular topic.

In some examples, a skill page may be created which shows skill-centricinformation relating to statistics and rankings of the particular skill.In some examples, the skill page may display a list of individualssorted by rank, a listing of top employers for the skill, a listing ofthe top geographic regions, a listing of the top groups for the skill onthe social networking site, or any other relevant information.

In still other examples, job postings may be customized for a memberbased upon their skill rank. In some examples, job postings may onlyappear to members above or below a certain skill rank, or that possess acertain skill. In some examples, job postings may be deliveredautomatically by the social or business network to members with aspecific rank or a rank exceeding or under a specific amount. In somecases, jobs may not be shown, delivered, or available to members thatrank too high in the rankings. This may be because employers do not wantsomeone too skilled and therefore expensive.

Job postings may be customizable based upon a combination of skills andrankings. Thus a job posting may be delivered or viewable only toindividuals possessing a requisite rank in multiple skills. Thus forexample, a job posting may require a member to be highly ranked in bothJava and C++.

In other examples, the system may deliver to a third party, such as ajob recruiter, a list of members who possess a particular skill orcombination of particular skills. In some examples, the system maydeliver to the third party a list of members who possess a requisiterank in the particular skill or combination of particular skills.

Additionally, advertisements may be customized and delivered to aparticular member based upon their skill rank in various skills. Forexample, an individual who ranks highly in C++ might receiveadvertisements directed at C++ compilers. These advertisements may evenbe tailored for a level of product based upon a member ranking. Forexample, an advertisement for an advanced version of the C++ compiler oran advanced programming textbook may be delivered to users that havehigher rankings, and advertisements for basic versions of the C++compiler or a basic programming textbook may be delivered to lowerranking users.

FIG. 13 shows an example system for implementing the skillscustomization. In FIG. 13, skill rankings 13010, profile information13020, and external information 13030 may be used as input into thecustomization process 13040. The customization process 13040 may includea skill reports process 13050, a skill advertisement process 13060, askill recommendation process 13070, and a job posting process 13080.

The skill reports process 13050 may be responsible for utilizing skillrankings 13010, profile information 13020, and external information13030 to prepare and display reports on the skill hierarchy, skillrankings, company or geographical rankings, or other reports.

The skill advertisement process 13060 may be responsible for deliveringadvertisements to members based upon their skill rankings. This mayinclude storing criteria for various advertisements. These criteria mayspecify conditions on which the advertisement will be displayed.Conditions in some examples may include an identification of a certainskill or skills that the member must possess prior to displaying theadvertisement to the member. In other examples, the conditions may alsoinclude a skill level that a member must have in order for theadvertisement to be displayed to the member. Thus for example, theconditions may specify that only members above a certain skill levelskilled in coding in the C++ computer language may receive anadvertisement for an advanced C++ compiler.

In one example, the skill advertisement process 13060 may find memberswho match the criteria, and then may be responsible for causing theadvertisement to be displayed to the members.

The skill recommendation process 13070 may be responsible forformulating a recommendation for an interested member on how to improvetheir skill ranking. The skill recommendation process 13070 may use theactivities of the interested member, other lower or higher rankedmembers, and knowledge of the ranking algorithm itself to suggestchanges in member behavior, additional activities, or additionalconnections that may increase the member's ranking. In some examplesthese recommendations may include connecting with certain members,working for a certain company, or living and working in a certaingeographic area, and the like.

The job postings process 13080 may be responsible for matching jobposting criteria with qualified members. The job posting criteria mayinclude a desired set of one or more skills that the employer isinterested in, and possibly a desired level of skill. The job postingprocess 13080 then matches job posting criteria with members that matchthat criteria and may then be responsible for delivering that jobposting to members.

Example Social Networking Service

FIG. 14 shows an example social networking service 14000 according toone example of the current disclosure. Social networking service 14000may contain a content server process 14010. Content server process 14010may communicate with storage 14090 and users 14060 through a network.Content server process 14010 may be responsible for the retrieval,presentation, and maintenance of member profiles stored in storage14090. Content server process 14010 in one example may include or be aweb server that fetches or creates internet web pages, which may includeportions of, or all of, a member profile at the request of users 14060.

Users 14060 may be an individual, group, or other member, prospectivemember, or other user of the social networking service 14000. Users14060 access social networking service 14000 using a computer systemthrough a network. The network may be any means of enabling the socialnetworking service 14000 to communicate data with a computer remotely,such as the internet, an extranet, a LAN, WAN, wireless, wired, or thelike, or any combination.

Skills process 14030 may be responsible for creating the list of skills,ranking members based upon the created list of skills and customizingthe social networking service 14000 based upon those rankings. Skillsprocess 14030 in one example may contain a skills extraction process14040 to create a list of skills based upon member profiles, a skillsranking process 14050 for ranking users relative to each other for eachskill in the list of skills, and a customization process 14060 whichuses the skills and rankings to customize the social networking service14000 for the members based upon the skill rankings.

Batch processing system 14020 may be a computing entity which is capableof data processing operations either serially or in parallel. In someexamples, batch processing system 14020 may be a single computer. Inother examples, batch processing system 14020 may be a series ofcomputers setup to process data in parallel. In some examples, batchprocessing system 14020 may be part of social networking service 14000.

Skills process 14030 may communicate with the social networking service14000 to get information used by the skills process 14030 such as memberprofiles or information from member profiles, and to customize thesocial networking service 14000 based upon the skills and theirrankings.

Skills process 14030 may also communicate with a crowdsourcingapplication 14080 and various external data sources 14070 across anetwork. The network may be any method of enabling communication betweensocial networking service 14000 and crowd sourcing application 14080and/or external data sources 14070. Examples may include, but are notlimited to, the internet, an extranet, a LAN, WAN, or wireless network.Skills process 14030 submits de-duplication jobs through the network tothe crowdsourcing application 14080 for de-duplication. Crowdsourcingapplication 14080 may return the results back over the network. Skillsprocess 14030 may also utilize a network to access various remote datasystems. The various described networks may be the same or differentnetworks.

Skills extraction process 14040 may extract a standardized list ofskills from the various member profiles as well as calculating thevarious statistics and meta data about those skills. Skills rankingprocess 14050 may rank members based on the provided skills.Customization process 14060 may customize the social networking service14000 based upon the skill rankings.

FIG. 15-FIG. 17 show screen captures of example user interfacesimplementing the customization techniques of the present disclosure. InFIG. 15, a supply index graph is shown that maps the number of LinkedInmembers who possess the skill “Hadoop” over a period of years(1990-present). FIG. 15 also shows links and functionality that allows amember to see other members that they are connected to that have“Hadoop” as a skill. FIG. 15 shows the number of members of the sitethat have “Hadoop” as a skill, and a ranking of companies and locationsbased upon the “Hadoop” skill. A list of related skills is shown, aswell as the top ranking members.

FIG. 16 shows a more complete view of the ranking of members who know“Hadoop,” as well as a number of member groups that possess the “Hadoop”skill.

FIG. 17 shows a listing of jobs that require a “Hadoop” skill, as wellas additional members with Hadoop and additional groups of Hadoop users.

Modules, Components, and Logic

Certain embodiments are described herein as including logic or a numberof components, modules, or mechanisms. Modules may constitute eithersoftware modules (e.g., code embodied (1) on a non-transitorymachine-readable medium or (2) in a transmission signal) orhardware-implemented modules. A hardware-implemented module is tangibleunit capable of performing certain operations and may be configured orarranged in a certain manner. In example embodiments, one or morecomputer systems (e.g., a standalone, client or server computer system)or one or more processors may be configured by software (e.g., anapplication or application portion) as a hardware-implemented modulethat operates to perform certain operations as described herein.

In various embodiments, a hardware-implemented module may be implementedmechanically or electronically. For example, a hardware-implementedmodule may comprise dedicated circuitry or logic that is permanentlyconfigured (e.g., as a special-purpose processor, such as a fieldprogrammable gate array (FPGA) or an application-specific integratedcircuit (ASIC)) to perform certain operations. A hardware-implementedmodule may also comprise programmable logic or circuitry (e.g., asencompassed within a general-purpose processor or other programmableprocessor) that is temporarily configured by software to perform certainoperations. It will be appreciated that the decision to implement ahardware-implemented module mechanically, in dedicated and permanentlyconfigured circuitry, or in temporarily configured circuitry (e.g.,configured by software) may be driven by cost and time considerations.

Accordingly, the term “hardware-implemented module” should be understoodto encompass a tangible entity, be that an entity that is physicallyconstructed, permanently configured (e.g., hardwired) or temporarily ortransitorily configured (e.g., programmed) to operate in a certainmanner and/or to perform certain operations described herein.Considering embodiments in which hardware-implemented modules aretemporarily configured (e.g., programmed), each of thehardware-implemented modules need not be configured or instantiated atany one instance in time. For example, where the hardware-implementedmodules comprise a general-purpose processor configured using software,the general-purpose processor may be configured as respective differenthardware-implemented modules at different times. Software mayaccordingly configure a processor, for example, to constitute aparticular hardware-implemented module at one instance of time and toconstitute a different hardware-implemented module at a differentinstance of time.

Hardware-implemented modules may provide information to, and receiveinformation from, other hardware-implemented modules. Accordingly, thedescribed hardware-implemented modules may be regarded as beingcommunicatively coupled. Where multiple of such hardware-implementedmodules exist contemporaneously, communications may be achieved throughsignal transmission (e.g., over appropriate circuits and buses) thatconnect the hardware-implemented modules. In embodiments in whichmultiple hardware-implemented modules are configured or instantiated atdifferent times, communications between such hardware-implementedmodules may be achieved, for example, through the storage and retrievalof information in memory structures to which the multiplehardware-implemented modules have access. For example, onehardware-implemented module may perform an operation, and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware-implemented module may then,at a later time, access the memory device to retrieve and process thestored output. Hardware-implemented modules may also initiatecommunications with input or output devices, and may operate on aresource (e.g., a collection of information).

The various operations of example methods described herein may beperformed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implemented modulesthat operate to perform one or more operations or functions. The modulesreferred to herein may, in some example embodiments, compriseprocessor-implemented modules.

Similarly, the methods described herein may be at least partiallyprocessor-implemented. For example, at least some of the operations of amethod may be performed by one or processors or processor-implementedmodules. The performance of certain of the operations may be distributedamong the one or more processors, not only residing within a singlemachine, but deployed across a number of machines. In some exampleembodiments, the processor or processors may be located in a singlelocation (e.g., within a home environment, an office environment or as aserver farm), while in other embodiments the processors may bedistributed across a number of locations.

The one or more processors may also operate to support performance ofthe relevant operations in a “cloud computing” environment or as a“software as a service” (SaaS). For example, at least some of theoperations may be performed by a group of computers (as examples ofmachines including processors), these operations being accessible via anetwork (e.g., the Internet) and via one or more appropriate interfaces(e.g., Application Program Interfaces (APIs).)

Electronic Apparatus and System

Example embodiments may be implemented in digital electronic circuitry,or in computer hardware, firmware, software, or in combinations of them.Example embodiments may be implemented using a computer program product,e.g., a computer program tangibly embodied in an information carrier,e.g., in a machine-readable medium for execution by, or to control theoperation of, data processing apparatus, e.g., a programmable processor,a computer, or multiple computers.

A computer program may be written in any form of programming language,including compiled or interpreted languages, and it may be deployed inany form, including as a stand-alone program or as a module, subroutine,or other unit suitable for use in a computing environment. A computerprogram may be deployed to be executed on one computer or on multiplecomputers at one site or distributed across multiple sites andinterconnected by a communication network.

In example embodiments, operations may be performed by one or moreprogrammable processors executing a computer program to performfunctions by operating on input data and generating output. Methodoperations may also be performed by, and apparatus of exampleembodiments may be implemented as, special purpose logic circuitry,e.g., a field programmable gate array (FPGA) or an application-specificintegrated circuit (ASIC).

The computing system may include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. Inembodiments deploying a programmable computing system, it will beappreciated that that both hardware and software architectures requireconsideration. Specifically, it will be appreciated that the choice ofwhether to implement certain functionality in permanently configuredhardware (e.g., an ASIC), in temporarily configured hardware (e.g., acombination of software and a programmable processor), or a combinationof permanently and temporarily configured hardware may be a designchoice. Below are set out hardware (e.g., machine) and softwarearchitectures that may be deployed, in various example embodiments.

Example Computer Architecture

FIG. 18 shows a diagrammatic representation of a machine in the exampleform of a computer system 18000 within which a set of instructions forcausing the machine to perform any one or more of the methods,processes, operations, or methodologies discussed herein may beexecuted. In alternative embodiments, the machine operates as astandalone device or may be connected (e.g., networked) to othermachines. In a networked deployment, the machine may operate in thecapacity of a server or a client machine in server-client networkenvironment, or as a peer machine in a peer-to-peer (or distributed)network environment. The machine may be a Personal Computer (PC), atablet PC, a Set-Top Box (STB), a Personal Digital Assistant (PDA), acellular telephone, a Web appliance, a network router, switch or bridge,or any machine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine. Further,while only a single machine is illustrated, the term “machine” shallalso be taken to include any collection of machines that individually orjointly execute a set (or multiple sets) of instructions to perform anyone or more of the methodologies discussed herein. Example embodimentsmay also be practiced in distributed system environments where local andremote computer systems which that are linked (e.g., either byhardwired, wireless, or a combination of hardwired and wirelessconnections) through a network, both perform tasks. In a distributedsystem environment, program modules may be located in both local andremote memory-storage devices (see below).

The example computer system 18000 includes a processor 18002 (e.g., aCentral Processing Unit (CPU), a Graphics Processing Unit (GPU) orboth), a main memory 18001 and a static memory 18006, which communicatewith each other via a bus 18008. The computer system 18000 may furtherinclude a video display unit 18010 (e.g., a Liquid Crystal Display (LCD)or a Cathode Ray Tube (CRT)). The computer system 18000 also includes analphanumeric input device 18012 (e.g., a keyboard), a User Interface(UI) cursor controller 18014 (e.g., a mouse), a disk drive unit 18016, asignal generation device 18018 (e.g., a speaker) and a network interfacedevice 18020 (e.g., a transmitter).

The disk drive unit 18016 includes a machine-readable medium 18022 onwhich is stored one or more sets of instructions 18024 and datastructures (e.g., software) embodying or used by any one or more of themethodologies or functions illustrated herein. The software may alsoreside, completely or at least partially, within the main memory 18001and/or within the processor 18002 during execution thereof by thecomputer system 18000, the main memory 18001 and the processor 18002also constituting machine-readable media.

The instructions 18024 may further be transmitted or received over anetwork 18026 via the network interface device 18020 using any one of anumber of well-known transfer protocols (e.g., HTTP, Session InitiationProtocol (SIP)).

The term “machine-readable medium” should be taken to include a singlemedium or multiple media (e.g., a centralized or distributed database,and/or associated caches and servers) that store the one or more sets ofinstructions. The term “machine-readable medium” shall also be taken toinclude any medium that is capable of storing, encoding, or carrying aset of instructions for execution by the machine and that cause themachine to perform any of the one or more of the methodologiesillustrated herein. The term “machine-readable medium” shall accordinglybe taken to include, but not be limited to, solid-state memories, andoptical and magnetic medium.

Method embodiments illustrated herein may be computer-implemented. Someembodiments may include computer-readable media encoded with a computerprogram (e.g., software), which includes instructions operable to causean electronic device to perform methods of various embodiments. Asoftware implementation (or computer-implemented method) may includemicrocode, assembly language code, or a higher-level language code,which further may include computer readable instructions for performingvarious methods. The code may form portions of computer programproducts. Further, the code may be tangibly stored on one or morevolatile or non-volatile computer-readable media during execution or atother times. These computer-readable media may include, but are notlimited to, hard disks, removable magnetic disks, removable opticaldisks (e.g., compact disks and digital video disks), magnetic cassettes,memory cards or sticks, Random Access Memories (RAMs), Read OnlyMemories (ROMs), and the like.

Additional Notes

The above detailed description includes references to the accompanyingdrawings, which form a part of the detailed description. The drawingsshow, by way of illustration, specific embodiments in which theinvention may be practiced. These embodiments are also referred toherein as “examples.” Such examples may include elements in addition tothose shown or described. However, the present inventors alsocontemplate examples in which only those elements shown or described areprovided. Moreover, the present inventors also contemplate examplesusing any combination or permutation of those elements shown ordescribed (or one or more aspects thereof), either with respect to aparticular example (or one or more aspects thereof), or with respect toother examples (or one or more aspects thereof) shown or describedherein.

All publications, patents, and patent documents referred to in thisdocument are incorporated by reference herein in their entirety, asthough individually incorporated by reference. In the event ofinconsistent usages between this document and those documents soincorporated by reference, the usage in the incorporated reference(s)should be considered supplementary to that of this document; forirreconcilable inconsistencies, the usage in this document controls.

In this document, the terms “a” or “an” are used, as is common in patentdocuments, to include one or more than one, independent of any otherinstances or usages of “at least one” or “one or more.” In thisdocument, the term “or” is used to refer to a nonexclusive or, such that“A or B” includes “A but not B,” “B but not A,” and “A and B,” unlessotherwise indicated. In this document, the terms “including” and “inwhich” are used as the plain-English equivalents of the respective terms“comprising” and “wherein.” Also, in the following claims, the terms“including” and “comprising” are open-ended, that is, a system, device,article, or process that includes elements in addition to those listedafter such a term in a claim are still deemed to fall within the scopeof that claim. Moreover, in the following claims, the terms “first,”“second,” and “third,” etc. are used merely as labels, and are notintended to impose numerical requirements on their objects.

The above description is intended to be illustrative, and notrestrictive. For example, the above-described examples (or one or moreaspects thereof) may be used in combination with each other. Otherembodiments may be used, such as by one of ordinary skill in the artupon reviewing the above description. The Abstract is provided to complywith 37 C.F.R. §1.72(b), to allow the reader to quickly ascertain thenature of the technical disclosure. It is submitted with theunderstanding that it will not be used to interpret or limit the scopeor meaning of the claims. Also, in the above Detailed Description,various features may be grouped together to streamline the disclosure.This should not be interpreted as intending that an unclaimed disclosedfeature is essential to any claim. Rather, inventive subject matter maylie in less than all features of a particular disclosed embodiment.Thus, the following claims are hereby incorporated into the DetailedDescription, with each claim standing on its own as a separateembodiment, and it is contemplated that such embodiments may be combinedwith each other in various combinations or permutations. The scope ofthe invention should be determined with reference to the appendedclaims, along with the full scope of equivalents to which such claimsare entitled.

1. A method comprising: retrieving from non-volatile storage a pluralityof member profiles created by a plurality of members of a socialnetworking service; executing, on one or more computer processors, atext classification algorithm to determine which of the plurality ofmembers possesses a skill that matches any of a plurality of providedskills and associated skill attributes; and for at least one skill ofthe plurality of provided skills, identifying the plurality of membersthat possess the skill and ranking the plurality of members relative toone another using a ranking algorithm, the ranking algorithm being basedin part upon weighted interactions among the plurality of members thatpossess the given skill, the weighted interactions comprisingendorsements between a first member who possesses the given skill and asecond member who possesses the given skill.
 2. The method of claim 1,wherein the associated skill attributes includes co-ocurrent phrases. 3.The method of claim 1, wherein the text classification algorithm is abayes classifier.
 4. The method of claim 3, wherein evidence used in thebayes classifier comprises the plurality of provided skills andassociated skill attributes.
 5. The method of claim 1, furthercomprising: collecting a plurality of member behavior information. 6.The method of claim 5, wherein the weighted interactions include theplurality of member behavior information.
 7. The method of claim 1,wherein the endorsements comprise: an invitation to connect sent by thefirst member to the second member.
 8. The method of claim 1, wherein theendorsements comprise: member profile views of the first member by thesecond member.
 9. The method of claim 1, wherein the endorsementscomprise: inclusion of the first member by the second member in thesecond member's address book.
 10. The method of claim 1, wherein theendorsements comprise: the first and second members appearing in acommon group on the social networking site.
 11. The method of claim 1,further comprising: calculating a score for a company for the givenskill, by aggregating a rank for the given skill of any of the pluralityof members who possess the given skill and who report in their profilesthat they work for the company; and based on the company score,increasing or decreasing the rank for the given skill of any of theplurality of members who possess the given skill and who report in theirprofiles that they work for the company.
 12. The method of claim 1,further comprising: adjusting a rank of a particular member selectedfrom the plurality of members who possess the given skill based uponconnections associated with the particular member on a second socialnetworking site.
 13. The method of claim 1, wherein a weight given to aparticular weighted interaction between the plurality of members whopossess the given skill is based upon a rank of the members involved inthe interaction.
 14. The method of claim 13 further comprising:iteratively adjusting the weights and recalculating the rankings untilconvergence.
 15. The method of claim 1, further comprising: calculatinga score for a geographic region for the given skill by aggregating arank for the given skill of any of the plurality of members who possessthe given skill and who report in their profiles that they work in thegeographic area; and based on the geographic score, increasing ordecreasing the rank for the given skill of any of the plurality ofmembers that possess the given skill and who report in their profilesthat they work in the geographic region.
 16. A system comprising: aretrieval module executable on a computer processor to retrieve aplurality of member profiles created by a plurality of members of asocial networking service; a tagging module executable on one or morecomputer processors to run a text classification algorithm on theplurality of member profiles to determine which of the plurality ofmembers possesses a skill that matches any of a plurality of providedskills and associated skill attributes; and a ranking module configuredto: for at least one skill of the plurality of provided skills, identifythe plurality of members that possess the skill and rank them relativeto each other using a ranking algorithm, the ranking algorithm beingbased at least upon weighted interactions among members that posses thegiven skill, the weighted interactions comprising endorsements between afirst member that possesses the given skill and a second member thatpossesses the given skill.
 17. The system of claim 16, wherein theassociated skill attributes includes co-occurrent phrases.
 18. Thesystem of claim 16, wherein the text classification algorithm is a bayesclassifier.
 19. The system of claim 18, wherein evidence used in thebayes classifier comprises the provided skills and associated skillattributes.
 20. The system of claim 16, wherein the tagging modulecollects a plurality of member behavior metrics.
 21. The system of claim20, wherein the rankings module adjusts the ranking of the members thatpossess the skill based upon the plurality of member behavior metrics.22. The system of claim 16, wherein the endorsements comprise: aninvitation to connect sent by the first member to the second member. 23.The system of claim 16, wherein the endorsements comprise: memberprofile views of the first member by the second member.
 24. The systemof claim 16, wherein the endorsements comprise: inclusion of the firstmember by the second member in the second member's address book.
 25. Thesystem of claim 16, wherein the endorsements comprise: the first andsecond members appearing in a common group on the social networkingsite.
 26. The system of claim 16, wherein the ranking module calculatesa score of a company for the given skill, by aggregating a rank for thegiven skill of any of the plurality of members who possess the givenskill and who report in their profiles that they work for the company;and based on the company score, increasing or decreasing the rank forthe given skill of any of the plurality of members that possess thegiven skill and who report in their profiles that they work for thecompany.
 27. The system of claim 16, wherein the ranking module adjustsa rank of a particular member selected from the plurality of membersthat possess the given skill based upon the number of connectionsassociated with the particular member on a second social networkingsite.
 28. The system of claim 16, wherein the ranking module adjustsweights given to a particular weighted interaction between the pluralityof members that possess the given skill based upon a rank of the membersinvolved in the interaction.
 29. The system of claim 28, wherein theranking module iteratively adjusts the weights and recalculates untilconvergence.
 30. The system of claim 16, wherein the ranking algorithmcalculates a score of a geographic region for the given skill, byaggregating a rank for the given skill of any of the plurality ofmembers who possess the given skill and who report in their profilesthat they work in the geographic area; and based on the geographicscore, increasing or decreasing the rank for the given skill of any ofthe plurality of members that possess the given skill and who report intheir profiles that they work in the geographic region.
 31. Amachine-readable storage medium including instructions, which whenexecuted on the machine, causes the machine to: retrieve fromnon-volatile storage a plurality of member profiles created by aplurality of members of a social networking service; execute, a textclassification algorithm to determine which of the plurality of memberspossesses a skill that matches any of a plurality of provided skills andassociated skill attributes; and for at least one skill of the pluralityof provided skills, identify the plurality of members that possess theskill and rank the plurality of members relative to one another using aranking algorithm, the ranking algorithm being based in part uponweighted interactions among the plurality of members that possess thegiven skill, the weighted interactions comprising endorsements between afirst member who possesses the given skill and a second member whopossesses the given skill.